Optimization for latency reduction in Product Quantization #397
Conversation
|
Thank you for your contribution! This PR looks really good. I have a couple of minor comments and suggestions. For helping with code readability, I have a few suggestions for placing the minInPlace in the following files:
No need to change the contents of the function or its signature, just its line placement. This way, it is closer to functions that are somewhat related. In line 218 of KMeansPlusPlusClusterer.java, it seems like zeroing the vector makes no difference as we are replacing every value in the next loop. Is this correct? |
Thanks for your suggestions, I will update accordingly. Yes, you are correct about line 218 of KMeansPlusPlusClusterer.java, for cleaner implementation had set to zero. Please let me know if you are of the opinion to remove it? |
|
@marianotepper @AbhijitKulkarni1 if this helps, I added an index construction and PQ benchmarks you can use to test the results. |
This PR optimizes the io.github.jbellis.jvector.quantization.KMeansPlusPlusClusterer#chooseInitialCentroids implementation using Java Vector API for the following cases.
Tested the optimized code with the setup as below and observed latency reduction of ~6% in Product Quantization for ada002-100k dataset execution.
Run Setup:
Jvector release used: 4.0.0-beta.1
JDK used:
openjdk version "23.0.1" 2024-10-15
OpenJDK Runtime Environment (build 23.0.1+11-39)
OpenJDK 64-Bit Server VM (build 23.0.1+11-39, mixed mode, sharing)
Socket 1 of m7i.metal-48xl machine used
To check the benefit of this optimization reasonably, following changes were done in Bench.java, PQParameters.java and Grid.java:
io/github/jbellis/jvector/example/Grid.java:141 --> /*indexes.forEach((features, index) -> { ...});*/